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(54) Secure virtual LANS 

(57) The present invention discloses a metliod for 
securely adding a new end station to a local area net- 
work (LAN) segmented into a number of virtual local ar- 
ea networks (VLANs). The invention is applicable to var- 
ious types of LANs such as Ethernet and token ring. The 
LAN comprises an authentication server (AS) which in- 
teracts with each new end station before connection to 
a VLAN Is allowed. The method Involves the AS admin- 
istering a test to the new end station, which may Involve 
prompting the new end station for a password or asking 
it to encrypt a given number using a secret algorithm 
known only to the new end station and to the AS. The 
AS examines the results of this test and detemriines 
whether the new end station is permitted to join the 
VLAN. For added security, the new end station can ver- 
ify authenticity of the AS by administering a test of its 
own, which may consist of prompting the AS for a pass- 
word of its own or asking it to encrypt a new number, the 
new end station subsequently determining whether the 
AS is indeed genuine before beginning to transmit any 
further information. In this way, an end station cannot 
join a VLAN without authentication by the AS and a le- 
gitimate end statton can verify whether the test it is 
asked to pass comes from a legitimate source, thereby 
avoiding network security breaches. 
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Description 

Field of tlie Invention 

[0001] This invention relates to local area networlcs, 
and specifically to a method for improving the security 
of information circulating within a virtual local area net- 
work. 

Background of the Invention 

[0002] Conventional local area networks (LANs) can 
be thought of as comprising a number of end stations 
(or terminals), connected to each other by a combination 
of links and switches. In addition, distant switches can 
be connected by virtual connections (VCs) passing 
through asynchronous transfer mode (ATM) switches. 
Such an extension of a LAN is often referred to as a LAN 
emulation over ATM (LANE) environment. As the 
number of end stations in the LAN or LANE environment 
grows, congestion of traffic and security issues become 
grave concerns of administrators of such networks. 
[0003] Segmentation of the LAN or LANE environ- 
ment into a number of virtual LANs (VLANs) has been 
used by network administrators to relieve traffic conges- 
tion and to provide security of information travelling with- 
in the network. The security provided by traditional 
VLANs Is based on two basic principles used for trans- 
mitting data packets within the network. For one, broad- 
cast and multteast traffic is transmitted only to end sta- 
tions that are members of the VLAN. In this case, a 
known broadcast or multicast address can be shared 
among intended recipients. Secondly, unicast traffic is 
transmitted only between the source and destination 
end stations, although the location of an intended recip- 
ient can often only be determined by first broadcasting 
a "discovery" packet to other end stations within the 
VLAN. Clearly, network security in the prior art is based 
on the premise that data is transmitted only to those end 
stations that are authorized to see the data, thereby 
avoiding security breaches due to inadvertent or mali- 
cious snooping by end stations outskJe the VLAN. A se- 
rious flaw In this approach is that end stations can join 
a VLAN with little or no authentication by the network. 
[0004] Membership in a VLAN can be defined by user 
name, access port identifier, end station media access 
control (MAC) address or Internet Protocol (IP) sub-net- 
work address. When membership in a VLAN is defined 
by access port identifier, a network administrator as- 
signs the physical ports (e.g. on an Ethernet switch or 
hub) that constitute elements of a VLAN. However, this 
does not prevent an intruder from disconnecting a legit- 
imate end statkMi and connecting an illegitimate one to 
the same physical port. Once connected, the illegitimate 
end station has access to possibly confidential informa- 
tion circulating within the VLAN. 
[0005] VLAN membership can also defined by refer- 
ring to a unique 48-bit MAC address that is assigned to 



each end station during manufacture. In this case, the 
networkadmlnistrator defines the MAC addresses of the 
end stations that constitute elements of the VLAN. 
When an end station is connected and begins transmit- 

5 ting data packets, the source MAC address contained 
in each data packet is used to determine the VLAN 
where the end station belongs. Unfortunately, this does 
not prevent an intruder from connecting an illegitimate 
end station to the network and inserting the MAC ad- 

10 dress of a legitimate end station into its data packets. 
Having successfully "emulated" a legitimate end station, 
the illegitimate end station gains access to restricted in- 
formation being communicated in the VLAN. 
[0006] Finally, the network administrator may also de- 

15 fine the 32-bit IP address blocks or user names of the 
end stations that are permitted to be members of the 
VLAN. The IP address and user name act similarly to 
the MAC address, and again, by inserting the identity of 
a legitimate end station Into its data packets, an illegiti- 

20 mate end station can gain access to restricted data. 
[0007] It would thus be of prime importance to provide 
a method of ensuring that unauthorized end stations 
cannot connect to a VLAN. Furthermore, in the case 
where an authentication mechanism would be provided 

25 to alleviate this difficulty, it would be beneficial to ensure 
that unauthorized switches cannot emulate such an au- 
thentication mechanism. 

Summan^ of the invention 

[COOS] It is an object of the present Invention to miti- 
gate or obviate one or more disadvantages of the prior 
art. 

[0009] Therefore, the inventton may be summarized 

35 in accordance with a first broad aspect a local area net- 
work, comprising a plurality of end stations and an au- 
thentication server, the LAN being segmented into a plu- 
rality of virtual local area networks (VLANs). each VLAN 
comprising at least one member end station, wherein 

40 the authentication server keeps track of which end sta- 
tions are members of which VLAN. keeps track of which 
end stations are authorized to join which VLAN and per- 
forms authentication of end stations joining a VLAN. 
[0010] The invention may be summarized in accord- 

45 ance with a second broad aspect as A local area net- 
work, comprising: a plurality of end stations; a plurality 
of LAN emulation servers (LESs); a LAN emulation con- 
figuration sen/er (LECS); and an authentication server 
(AS); the LAN being segmented into a plurality of virtual 

SO local area networks (VLANs). each VI^N comprising a 
respective LES and at least one member end station, 
each LES keeping track of which end stations are mem- 
bers in the respective VLAN, the LECS keeping track of 
which end stations are members of which VLAN; where- 

55 in the authentication sewer keeps track of which end 
stations are authorized to join which VLAN and performs 
authentication of end stations joining a VLAN. 
[0011] The invention may be summarized in accord- 
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ance with a third broad aspect as a method for securely 
adding a new end station to a local area networi< (U\N), 
the LAN comprising a plurality of end stations and an 
authentication server (AS), the LAN being segmented 
into a plurality of virtual local area networi<s (VLANs), 
each VLAN comprising at least one member end station, 
wherein the authentication server keeps track of which 
end stations are members of which VLAN, keeps track 
of which end stations are permitted to join which VLAN 
and performs authentication of end statbns joining a 
VLAN, the method comprising the steps of: the new end 
station sending to the AS a message identifying both the 
new end station and a desired VLAN; the new end sta- 
tion taking a first authentication test; and upon success- 
ful authentication of the new end station, the AS sending 
to the new end station a message indicating that the new 
end station is permitted to join the desired VLAN. 
[0012] The invention may be summarized in accord- 
ance withafourth broad aspect as a method for securely 
adding a new end station to a local area network (LAN), 
the LAN comprising a plurality of end stations, a plurality 
of LAN emulation servers (LESs), a LAN emulation con- 
figuration server (LEGS) and an authentication server 
(AS), each switch communicating with at least one end 
station, the new end station being connected to a switch, 
the LAN being segmented into a plurality of virtual local 
area networks (VLANs), each VLAN comprising a re- 
spective LES and at least one member end station, each 
LES keeping track of which end stations are currently 
members in the respective VLAN, the LECS keeping 
track of which end stations are permitted to be members 
of which VLAN, wherein the authentication sender per- 
forms authentication of end stations joining a VLAN, the 
method comprising the steps of: the new end station 
sending to its switch a message identifying both the new 
end station and a desired VLAN; the switch sending to 
the LECS a message requesting identity of the LES cor- 
responding to the desired VLAN; the LECS sending to 
the AS a message requesting authentication of the new 
end station; the AS generating a first encrypted number 
using a plain number and an algorithm known to the AS 
and to the new end station; the AS sending to the LECS 
a message comprising the plain number and the first en- 
crypted number; the LECS sending to the switch a mes- 
sage comprising the plain number; the switch sending 
to the new end station a message comprising the plain 
number; the new end station generating a second en- 
crypted number using the plain number and the algo- 
rithm; the new end station sending to the switch a mes- 
sage comprising the plain number and the second en- 
crypted number; the switch sending to the LECS a mes- 
sage comprising the plain number and the second en- 
crypted number; the LECS comparing the first encrypt- 
ed number to the second encrypted number; the LECS 
sending to the LES corresponding to the desired VLAN 
a message indicating that the new end station intends 
to join the desired VLAN; the LECS sending to the switch 
a message comprising identity of the LES correspond- 



ing to the desired VLAN; the switch sending to the LES 
corresponding to the desired VLAN a message request- 
ing that the new end station join the desired VLAN; and 
the LES corresponding to the desired VLAN sending to 
s the switch a message indicating that the new end station 
is allowed to join the desired VLAN. 

Brief Description of the Drawings 

10 [0013] The preferred embodiment of the present in- 
vention will now be described with reference to the at- 
tached drawings, in which: 

FIGURE 1 is a block diagram of a prior art LANE 

IS environment; 

FIGURE 2 isablockdiagramof asecure LANE en- 
vironment including two virtual local area networks, 
in accordance with the preferred embodiment of the 
present invention; 

20 FIGURE 3 is a message flow diagram representing 
end station authentication in the network of FIG- 
URE 2; and 

FIGURE 4 is a message flow diagram representing 
end station and network authentication in the net- 
25 work of FIGURE 2. 

Detailed Description of the Preferred Embodiment 

[0014] FIGURE 1 shows a local area network 50 com- 

30 prising a plurality of interconnected end stations 
101,102,105,106 such as personal computers, work- 
group servers or mainframe computers. Although for Il- 
lustrative purposes the network is assumed to be an 
Ethernet LAN, the present Invention applies equally well 

35 to other types of LANs, e.g., token ring, high-level data 
link control (HDLC) and AppleTalk. 
[001 5] In an Ethernet local area network, a frame sent 
by a transmitting end station in the LAN contains a head- 
er identifying the transmitting end station and an intend- 

40 ed recipient end station (using, e.g., source and desti- 
nation MAC addresses), as well as information to be ex- 
changed. The Ethernet frames can be transmitted using 
the Carrier Sense Multiple Access with Collision Detec- 
tion (CSMA-CD) protocol or any other media-access 

45 control protocol known or used in the art. In FIGURE 1 , 
the two end stations 101,102 sharing an Ethernet link 
201 can communicate with each other without additional 
interfacing, as any frame transmitted on a given shared 
link is "seen" by all end stations connected to that link. 

50 [0016] An Ethernet switch 301 connects multiple 
Ethernet links 201 ,202 and enables communication be- 
tween end stations appearing on the various Ethernet 
links. The Ethernet links 201,202 emanate from the 
Ethernet switch 301 In a star arrangement and the 

55 Ethemet switch keeps track of which end stations are 
connected to which link. When an Ethemet frame is re- 
ceived by the Ethernet switch 301 , it examines the head- 
er and transmits the frame over the Ethemet link con- 
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nected to the intended recipient; the frame is not trans- 
mitted to any of the other linl<s, thereby reducing traffic 
congestion on the Ethemet links. In some Instances, 
end stations may be connected to their Ethemet switch 
with a dedicated Ethernet link to avoid sharing the link 
bandwidth with other end stations, thus providing the 
end station with the maximum possible performance. An 
example of this is end stations 105,106 connected to 
Ethernet switch 303 by dedicated links 205,206. 
[001 7] I n a large local area network, it may be neces- 
sary to introduce several Ethemet switches in order to 
further reduce congestion on Ethemet links. Although 
there a number of mechanisms for interconnecting 
Ethernet switches in a network, LAN emulation over 
ATIVI (l-ANE) represents a common approach. Ethernet 
switches 301 ,303 communicate via virtual connections 
(VCs) through an ATM network consisting of an ATM 
switch 401 , in addition to ATM links 501 ,503 joining the 
Ethernet switches to the ATM switches. In a more com- 
plex network, there may be several ATM switches inter- 
connected by additional ATM links. 
[001 8] The Ethernet switches keep track of which end 
stations are connected to which of its local Ethernet 
links, and also knows which end stations are connected 
to other Ethernet switches in the network. A LAN emu- 
lation sen/er (LES) 602, connected to the network by an 
ATM link 505, comprises an updated table indicating 
which end stations are connected to which Ethernet 
switches, so that information contained in the Ethernet 
switches 301 ,303 may be kept up-to-date. A LAN emu- 
lation configuration sewer (LEGS) 601, responsible for 
'configuring" the network to which it is connected by an 
ATM link 504, handles initial connection of new end sta- 
tions into the LANE environment. 
[0019] When an Ethernet frame is received from an 
end station by an Ethemet switch, the header will be ex- 
amined and if the recipient is connected to one of its 
local Ethernet links, the Ethernet switch transmits the 
frame overthe appropriate Ethernet link. If, however the 
recipient is connected to another Ethemet switch, the 
frame is transmitted over the appropriate ATM VC to the 
destination Ethernet switch. Upon receipt of the frame 
over the ATM VC, the destination Ethernet switch per- 
forms a normal match of destination MAC address to 
Ethernet link and forwards the frame over the appropri- 
ate Ethemet link to the destination end station. 
[0020] If there are too many end stations in a LAN, 
multicast and broadcast traffic can become major con- 
tributors to network congestion. To alleviate this prob- 
lem, the network is segmented into a number of smaller, 
"Virtual" sub-networks (virtual LANS, or VLANs). As hint- 
ed at by the term 'virtual', end stations designated as 
belonging to a particular VLAN do not all have to connect 
to the one Ethernet switch nor do all end stations con- 
nected to an Ethernet switch have to belong to the one 
VLAN. Such partitioning of the network is transparent to 
the end stations. Each Ethernet switch, on the other 
hand, comprises an internal database to keep track of 



which end stations belong to which VLANs. 
[0021] In FIGURE 2 is shown an exemplary LANE en- 
vironment 50 in accordance with the present invention. 
Two virtual LANs can be identified: a "red" VLAN, con- 

5 sisting of end stations 101 -R, 102-R and 105-R, and a 
"green" VLAN, consisting of end stations 103-G, 104-G 
and 106-G. Other groups of end stations 108, 109 do 
not belong to either VLAN. Physically, end stations 
101-R and 102-R share an Ethernet link 201 and are 

10 connected to an Ethernet switch 301. From Ethernet 
switch 301 also emanates an Ethernet Iink202 connect- 
ing end stations 108. Similarly, an Ethernet switch 302 
connects end stations 103-G and 104-G via a shared 
Ethemet link 203 and end stations 109 via another 

IS Ethemet link 204. A third Ethernet switch 303 connects 
end stations 105-R and 106-G via respective dedicated 
Ethemet links 205 and 206. Ethernet switch 303 also 
physically connects an end station 107-R via a dedicat- 
ed Ethernet link 207. The end station 107-R is not a 

20 member of either the red or the green VLAN, but pre- 
sumably intends to join the red VLAN. 
[0022] A LAN emulation configuration sen/er (LEGS) 
601 contains an internal database storing a record of 
each VLAN and the end stations permitted to join the 

25 VLANs. As end stations are powered on or reconfigured, 
the Ethernet switches register the end stations wishing 
(and permitted) to join a particular VLAN with a LAN em- 
ulation sen/er (LES, 602-R for the red VLAN and 603-G 
for the green VLAN); registration with an LES consti- 

30 tutes membership within the corresponding VLAN. Vir- 
tual connections joining the Ethemet switches 301 , 302, 
303, the LEGS 601 and the LAN emulation sen/ers 
602-R, 602-G are established by an ATM switch or hub 
401, and communication is effected via ATM links 501 

35 through 506, respectively. 

[0023] A multicast or broadcast frame received from 
an end station that is a member of, for example, the red 
VLAN, is fonwarded by the Ethernet switch serving the 
end station to a broadcast and unknown server (BUS) 

40 function associated with LES 602-R. The LES 602-R 
then fonwards the frame to all Ethemet switches in the 
network that have end stations that are members of the 
red VLAN, i.e., Ethernet switches 301 and 303. The 
Ethemet switches 301 and 303 in turn forward the mul- 

45 ticast or broadcast frame only to those Ethernet links 
that are connected to members of the red VLAN, i.e., 
Ethemet links 201 and 205. In this way, multicast and 
broadcast frames are prevented from being transmitted 
to end stations outside the VLAN where the frame orig- 

50 inated, thereby relieving traffic congestion within the 
LAN as a whole. 

[0024] A primary function of the LEGS 601 is to con- 
figure the VLANs, i.e., to inform new end stations wish- 
ing to join a particular VLAN of the address where the 
ss LES for that VLAN can be found. In conventional net- 
works, however, no authentication of the new end sta- 
tions is performed. By using, say. the MAC address of 
an end station permitted to join a particular VLAN, a pos- 
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sibly unauthorized end station can register with the 
VU^N's LES, leading to the previously discussed secu- 
rity breaches. 

[0025] In accordance with the present invention, an 
authentication server (AS) 701, connected to the net- 
work via an ATIVI Iink507, provides security mechanisms 
for authenticating end stations when they attempt to join 
a desired VLAN. The AS 701 , for its part, is responsible 
for checking the validity of new end stations and not let- 
ting them register with any LES unless they pass an au- 
thentication 'test", which in an exemplary embodiment 
is administered using a key-based challenge-response 
algorithm. A network administrator can easily ensure 
that only the AS 701 and one new end station at a time 
possess appropriate keys for administering and passing 
the test. It is within the scope of the present invention to 
provide different types of authentication tests, such as 
techniques based on passwords, synchronized security 
cards, voice printing or finger printing. The key consid- 
eration in all cases is that successful authentication is 
possible only if the new end station is genuinely author- 
ized to join the desired VLAN. 
[0026] If the AS 701 is connected to the network 
through an Ethernet link and switch, the AS should not 
share its Ethernet link with other end stations to ensure 
that traffic directed to the AS is seen only by the AS. The 
AS may be implemented as a stand-alone entity to pro- 
vide enhanced security for the algorithms and data it 
contains, or may be integrated with the LEGS 601 . 
[0027] A sequence of steps for end station 107-R to 
join the red VLAN according to the present Invention is 
now described with additional reference to FIGURE 3, 
in which only the steps requiring transmission of infor- 
mation between network components have been illus- 
trated. It is to be understood that an analogous algorithm 
applies in the case of a new end station wishing to join 
the green VLAN. 

Step A. End station 107-R constructs an Ethernet 
frame consisting of a frame header compris- 
ing a destination address and a source ad- 
dress (e.g., the MAC address of end station 
107-R), as well as data to be exchanged. 
The destination address may be the MAC 
address of the destination end terminal or a 
known broadcast address. 

Step B. End station 107-R transmits the frame over 
Ethemet link 207 to Ethemet switch 303 in 
the form of a "Data" message, using the CS- 
MA-CD protocol. 

Step C. Ethemet switch 303 extracts the source ad- 
dress (the MAC address of end station 
1 07-R) from the Ethernet frame and consults 
an internal table to determine the virtual LAN 
(and LES) associated with the source ad- 
dress. 

Step D. If Ethernet switch 303 cannot find an asso- 
ciated LES by consulting its intemal table. 



Ethernet switch 303 sends a query, in the 
form of a "ConfigRqst" message, to the 
LECS 601 asking for the identity of the LES 
associated with end station 107-R. 

s StepE. LECS 601 sends an "Authenticate" mes- 
sage to AS 701 requesting authentication of 
end station 107-R. 
Step F. Using a challenge-response authentication 
algorithm, AS 701 generates a plain number, 

10 such as a random number RN, and encrypts 

it using a secret key known only to the AS 
701 and end station 1 07-R to produce E-RN. 
Both RN and E-RN are returned to the LECS 
601 as a "DoChallenge" message. The se- 

is cret key used to generate E-RN is never re- 

vealed by the AS 701 . 
Step G. The LECS 601 creates a frame containing a 
challenge to end station 107-R that includes 
RN received from AS 701 but does not in- 

20 elude E-RN. The frame is then sent in a 

■Challenge" message from the LECS 601 to 
the Ethernet switch 303 and subsequently 
relayed to end station 107-R. 
Step H. End station 107-R encrypts RN received in 

25 the challenge using its secret key and the 

same authentication algorithm used by the 
AS 701 . 

Step I. End station 107-R responds to the challenge 
with a "ChallengeResponse" message con- 
30 taining RN received from the LECS 601, 

along with its version of E-RN. The challenge 
response is relayed by Ethernet switch 303 
to the LECS 601. 
Step J. The LECS 601 compares the value of E-RN 
35 received from end station 1 07-R to the value 

of E-RN received earlier from the AS 701 . 

If the values match: 

40 step K. The LECS 601 consults its own internal ta- 
bles to determine that end station 107-R is 
associated with the red VLAN managed by 
LES 602-R. LECS 601 sends a "Notify" mes- 
sage to LES 602-R indicating that end sta- 

45 tion 107-R is attempting tojoin the red VLAN; 

this indication includes the MAC address of 
end station 107-R. 
Step L. The LECS 601 then sends the identity of 
LES 602-R in a "ConfigResp" message, re- 

SO sponding to the original query from Ethernet 

switch 303 at step D. 
Step M. If it does not currently have an ATM virtual 
connection to LES 602-R, Ethemet switch 
303 creates such a connection through ATM 

55 switch 401 using standard ATM signalling 

techniques. Ethernet switch 303 then sends 
a "JoinRqst" message for end station 107-R 
over this virtual connection to LES 602-R. 
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Step N, Upon receipt of this registration message, 

LES 602-R enters the MAC address of end Step B. 
station 107-R into its Internal tables and 
records the identity of Ethernet switch 303 
as the switch serving end station 1 07-R. LES s 
602-R sends a "JoinAck" message to Ether- Step C. 
net switch 303 acknowledging successful 
registration of end station 107-R as a mem- 
ber of the red VLAN. 

Step O. When Ethernet switch 303 receives the ac- 'O 
knowledgementto its registration request, it 
updates its internal tables to associate end Step D. 
station 107-R with the red VLAN managed 
by LES 602-R. 

IS 

If the values do not match: 

Step K'. The LEGS 601 sends a response to Ether- Step E. 
net switch 303 indicating that network ac- 
cess is denied to end station 107-R (not 20 
shown). Step F. 

Step L'. Ethemet switch 303 discards all frames re- 
ceived from end station 1 07-R and does not 
forward any frames to end station 107-R, 
thus isolating end station 107-R from the 
network. 

[0028] A second form of security attack involves a bo 
gus Ethernet switch that attempts to extract Information 
from a network by posing as a LAN emulation configu- so step G. 
ration sen/er or as an authentication server For exam- 
ple, if the above procedures are followed by a new end 
station genuinely authorized to enter the red VLAN, the 
bogus Ethernet switch can, without actually comparing 
the encrypted random numbers, pretend to give the new 3S 
end station permission to enter the red VLAN. From the 
new end station's point of view, having expected to be Step l-l. 
"let in" from the start, it begins an ejoihange of restricted 
infomiation that is now intercepted by the bogus Ether- 
net switch. 40 
[0029] To counter this attack, the new end station Step I. 
may, upon responding to the challenge issued by the 
network, administer its own test to verify authenticity of 
the issuer of the original challenge. Considering the net- 
work of FIGURE 2 and with reference to FIGURE 4, the 45 step J. 
following sequence of steps not only provides network 
security by verifying legitimacy of a new end station 
1 07-R upon entering the network, but allows (legitimate) 
new end station 107-R to protect itself from bogus test 
administrators. SO 

Step A. End station 107-R constructs an Ethernet StepK. 
frame consisting of a frame header compris- 
ing a destination address and a source ad- 
dress (e. g. , the MAC address of end station 55 
107-R), as well as data to be exchanged. 
The destination address may be the MAC 
address of the destination end terminal or a 



known broadcast address. 
End station 107-R transmits the frame over 
Ethernet link 207 to Ethemet switch 303 in 
the form of a "Data" message, using the CS- 
MA-CD protocol. 

Ethernet switch 303 extracts the source ad- 
dress (the MAC address of end station 
1 07-R) from the Ethernet frame and consults 
an internal table to determine the virtual LAN 
(and LES) associated with the source ad- 
dress. 

If Ethemet switch 303 cannot find an asso- 
ciated LES by consulting its internal table, 
Ethernet switch 303 sends a query, In the 
form of a "ConflgRqst" message, to the 
LECS 601 asking for the identity of the LES 
associated with end station 107-R. 
LECS 601 sends an "Authenticate" mes- 
sage to AS 701 requesting authentication of 
end station 107-R. 

Using a challenge-response authentication 
algorithm, AS 701 generates a plain number, 
such as a random number RN, and encrypts 
it using a secret key known only to the AS 
701 and end station 1 07-R to produce E-RN. 
Both RN and E-RN are returned to the LECS 
601 as a "DoChallenge" message. The se- 
cret key used to generate E-RN is never re- 
vealed by the AS 701 . 
The LECS 601 creates a frame containing a 
challenge to end station 107-R that includes 
RN received from AS 701 but does not in- 
clude E-RN. The frame is then sent in a 
•Challenge" message from the LECS 601 to 
the Ethernet switch 303 and subsequently 
relayed to end station 107-R. 
End station 107-R encrypts RN received in 
the challenge using its secret key and the 
same authentication algorithm used by the 
AS 701. 

End station 107-R generates a second plain 
number, such as a random number RN2, 
and encrypts it using its secret key to pro- 
duce E-RN2. 

End station 1 07-R responds to the challenge 
with a "ChallengeResponse" message that 
includes RN received from LECS 601 , along 
with RN2 and its version of E-RN, but does 
not include E-RN2. The challenge response 
is relayed by Ethernet switch 303 to LECS 
601. 

After first ensuring that end statkjn 107-R is 
legitimate by verifying that the value of E-RN 
received from end station 1 07-R matches 
the value of E-RN received from AS 701, 
LECS 601 sends a "Challenge" message to 
AS 701 that includes RN2 and the MAC ad- 
dress of end station 107-R. 
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Step L. AS 701 encrypts RN2 received in the clial- 
lenge using the authientication algorithm and 
the secret Icey for end station 1 07-R and re- 
turns its version of E-RN2 to LEGS 601 in 
the form of a "ChaliengeResponse" mes- 



Step M. LEGS 601 consults its own internal tables to 
determine that end station 107-R is associ- 
ated with the red VLAN managed by LES 
602-R. LEGS 601 sends a "Notify" message 
LES 602-R indicating that end station 107-R 
is attempting to join the red Vl^N; this indi- 
cation includes the MAG address of end sta- 
tion 107-R, the random number RN2 re- 
ceived in the challenge from end station 
1 07-R and the encrypted random number E- 
RN2 calculated by AS 701 . 

Step N. The LEGS 601 then sends the identity of 
LES 602-R in a "GonfigResp" message, re- 
sponding to the original query from Ethernet 
switch 303 at Step D. 

Step O. If it does not currently have an ATM virtual 
connection to LES 602-R, Ethernet switch 
303 creates such a connection through ATM 
switch 401 using standard ATM signalling 
techniques. Ethernet switch 303 then sends 
a "JoinRqst" message for end station 107-R 
over this virtual connection to LES 602-R. 

Step P. Upon receipt of this registration message, 
LES 602-R enters the MAC address of end 
station 107-R Into its internal tables and 
records the identity of Ethernet switch 303 
as the switch sen/ing end station 1 07-R. LES 
602-R sends a "JoinAck" message to Ether- 
net switch 303 acknowledging successful 
registration of end station 107-R as a mem- 
ber of the red VLAN. 

Step Q. When Ethernet switch 303 receives the ac- 
knowledgement to its registration request, it 
updates its internal tables to associate end 
station 107-R with the red VLAN managed 
by LES 602-R. 

Step R. Using the information received from LEGS 
601 , LES 602-R also sends a "ChaliengeRe- 
sponse" message to end station 107-R, via 
Ethernet switch 303, that includes the ran- 
dom number RN2 generated by end station 
107-R and the encrypted random number E- 
RN2 calculated by the AS 701 . 

Step S. When the challenge response is received, 
end station 1 07-R compares the value of E- 
RN2 received from LES 602-R with the value 
computed locally. If the values match, end 
station 107-R is assured that the network 
connection is legitimate. 

[0030] It is to be understood that alternate embodi- 
ments of the present invention exist in which ATM 



switches are not empbyed, eliminating any requirement 
for a LAN emulation configuration sen/er or LAN emu- 
lation servers. In such a case, a specific member of each 
VLAN would be designated as the "VLAN server" and 
s configuration of the network could easily be relegated 
to the authentication sen/er The entire authentication 
procedure could be accomplished by communication 
between the authentication server and the designated 
Vl-AN servers. 

10 [0031] While the preferred embodiment of the inven- 
tion has been described and illustrated it will be appar- 
ent to one skilled in the art that variations in the design 
may be made. The scope of the invention, therefore, is 
only to be limited by the claims appended hereto. 



Claims 

1. A local area network, comprising a plurality of end 
20 stations and an authentication server, the LAN be- 
ing segmented into a plurality of virtual local area 
networks (VLANs), each VLAN comprising at least 
one member end station, wherein the authentica- 
tion sen/er keeps track of which end stations are 

25 members of which VLAN, keeps track of which end 
stations are authorized to join which VLAN and per- 
forms authentication of end stattons joining a VLAN. 

2. A local area network, comprising: 

30 

a plurality of end stations; 

a plurality of LAN emulation servers (LESs); 

a LAN emulation configuration server (LEGS) ; 

and 

3S an authentication server (AS) ; 

the LAN being segmented into a plurality of vir- 
tual local area networks (VLANs), each VLAN 
comprising a respective LES and at least one 
member end station, each LES keeping track 

40 of which end stations are members in the re- 

spective VLAN, the LEGS keeping track of 
which end stations are members of which 
VLAN; 

wherein the authentication sen/er keeps track 
45 of which end stations are authorized to join 

which VLAN and performs authentication of 
end stations joining a VLAN. 

3. A local area network of claim 2, wherein the LEGS 
so Is merged with the AS. 

4. A local area network as claimed in any of claims 1 
to 3 being a token ring LAN. 

ss s. A local area network as claimed in any of claims 1 
to 3 being an Ethernet LAN. 

6. A local area network as claimed in claim 5 further 
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comprising a plurality of Etiiernet switches, each 
switch communicating with at least one end station 
through an Ethernet communication linl<. 

7. A method for securely adding a new end station to 
a local area networic (LAN), the LAN comprising a 
plurality of end stations and an authentication sen/- 
er (AS), the LAN being segmented into a plurality 
of virtual local area networl<s (VLANs), each VLAN 
comprising at least one member end station, where- 
in the authentication server keeps track of which 
end stations are members of which VLAN, keeps 
track of which end stations are permitted to join 
which VLAN and performs authentication of end 
stations joining a VLAN, the method comprising the 
steps of: 

the new end station sending to the AS a mes- 
sage identifying both the new end station and 
a desired VLAN; 

the new end station taking a first authentication 
test; and 

upon successful authentication of the new end 
station, the AS sending to the new end station 
a message Indicating that the new end station 
Is permitted to join the desired VLAN. 

8. A method as claimed In claim 7, further comprising 
the steps of: 

the new end station sending to the AS a mes- 
sage identifying both the new end station and 
a desired VLAN; 

the AS station taking a second authentication 
test; and 

upon successful authentication of the AS, the 
new end station joining the desired VLAN. 

9. A method as claimed in claim 8, wherein the new 
end station stores a second list of passwords and 
the second authentication test consists of: 

the AS comparing sending a message to the 
new end station comprising a third password; 

and 

the new end station comparing the third pass- 
word to a fourth password contained in the sec- 
ond list of passwords; 

wherein authentication of the AS is said to have 
been successful if the third and fourth pass- 
words are Identical. 



new end statbn; 

the new end station sending to the AS a mes- 
sage comprising the second plain number; 
the AS generating a fourth encrypted number 
s using the second plain number and the second 

algorithm; 

the AS sending to the new end station a mes- 
sage comprising the second plain number and 
the fourth encrypted number; and 
10 the new end station comparing the third en- 

crypted number to the fourth encrypted 
number; 

wherein authentication of the AS is said to have 
been successful if the third and fourth encrypt- 
is ed numbers are identical. 

1 1 . A method as claimed in any of claims 7 to 1 0, where- 
in the AS stores a first list of passwords and the first 
authentication test consists of: 

20 

the new end station sending a message to the 
AS comprising a first password; and 
the AS comparing the first password to a sec- 
ond password contained in the first list of pass- 
as words; 

wherein authentication of the new end station 
is said to have been successful if the first and 
second passwords are identical. 

30 12. A method as claimed in any of claims 7 to 10, where- 
in the first authentication test consists of: 

the AS generating a first encrypted number us- 
ing a plain number and a first algorithm known 

3S to the AS and to the new end station; 

the AS sending to the new end station a mes- 
sage comprising the plain number; 
the new end station generating a second en- 
crypted number using the plain number and the 

40 first algorithm; 

the new end station sending to the AS a mes- 
sage comprising the plain number and the sec- 
ond encrypted number; and 
the AS comparing the first encrypted number to 

45 the second encrypted number; 

wherein authentication of the new end station 
is said to have been successful If the first and 
second encrypted numbers are identical. 

50 13. A method as claimed in claim 12, wherein the first 
plain number Is a random number 



10. A method as claimed in claim 8, wherein the second 
authentication test consists of: 

the new end station generating a third encrypt- 
ed number using a second plain number and a 
second algorithm known to the AS and to the 



14. A method as claimed in claim 12, wherein the first 
algorithm is a key-based encryption algorithm. 

SB 

1 5. A method for securely adding a new end station to 
a local area network (LAN), the LAN comprising a 
plurality of end stations, a plurality of LAN emulation 
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servers (LESs), a LAN emulation configuratton 
server (LEGS) and an authentication server (AS), 
eachi switch communicating with at least one end 
station, the new end station being connected to a 
switch, the LAN being segmented into a plurality of s 
virtual local area ne1worl<s (VLANs), each VLAN 
comprising a respective LES and at least one mem- 
ber end station, each LES keeping tracl< of which 
end stations are currently members in the respec- 
tive VLAN, the LEGS keeping track of which end 'o 
stations are permitted to be members of which 
VLAN, wherein the authentication server performs 
authentication of end stations joining a VLAN, the 
method comprising the steps of: 

IS 

the new end station sending to its switch a mes- 
sage identifying both the new end station and 
a desired VLAN; 

the switch sending to the LEGS a message re- 
questing identity of the LES corresponding to so 
the desired VLAN; 

the LEGS sending to the AS a message re- 
questing authentication of the new end station; 
the AS generating a first encrypted number us- 
ing a plain number and an algorithm known to 2S 
the AS and to the new end station; 
the AS sending to the LEGS a message com- 
prising the plain number and the first encrypted 
number; 

the LEGS sending to the switch a message 30 
comprising the plain number; 
the switch sending to the new end station a 
message comprising the plain number; 
the new end station generating a second en- 
crypted number using the plain number and the as 
algorithm; 

the new end station sending to the switch a 
message comprising the plain number and the 
second encrypted number; 
the switch sending to the LEGS a message 40 
comprising the plain number and the second 
encrypted number; 

the LEGS comparing the first encrypted 

number to the second encrypted number; 

the LEGS sending to the LES corresponding to 4S 

the desired VLAN a message indicating that the 

new end station intends to join the desired 

VLAN; 

the LEGS sending to the switch a message 
comprising identity of the LES corresponding to so 
the desired VLAN; 

the switch sending to the LES corresponding to 
the desired VLAN a message requesting that 
the new end station join the desired VLAN; and 
the LES corresponding to the desired VLAN ss 
sending to the switch a message indicating that 
the new end station is allowed to join the de- 
sired VLAN. 



1 6. A method as claimed in any of claims 7 to 1 5, where- 
in the new end station is identified by a 48-blt media 
access control address. 

1 7. A method as claimed in any of claims 7 to 1 5, where- 
in the new end station is identified by a 32-bit Inter- 
net Protocol address. 

1 8. A method as claimed in any of claims 7 to 1 5, where- 
in the new end station is identified by a physical port 
on an Ethernet switch. 
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Fig. 2 
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Fig. 3 

station 107-R Switch 303 LEGS 601 LES602-R AS 701 
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Fig. 4 

station 107-R Switch 303 LEGS 601 LES602-R AS 701 
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